On Training Neural Networks with Mixed Integer Programming

Recent work has shown potential in using Mixed Integer Programming (MIP) solvers to optimize certain aspects of neural networks (NN). However little research has gone into training NNs with solvers. State of the art methods to train NNs are typically gradient-based and require significant data, computation on GPUs and extensive hyper-parameter tuning. In contrast, training with MIP solvers should not require GPUs or hyper-parameter tuning but can likely not handle large amounts of data. This work builds on recent advances that train binarized NNs using MIP solvers. We go beyond current work by formulating new MIP models to increase the amount of data that can be used and to train non-binary integer-valued networks. Our results show that comparable results to using gradient descent can be achieved when minimal data is available

Progressing intention progression: a call for a Goal-Plan Tree contest

User-supplied domain control knowledge in the form of hierarchically structured Goal-Plan Trees (GPTs) is at the heart of a number of approaches to reasoning about action. Reasoning with GPTs connects the AAMAS community with other communities such as automated planning, and forms the foundation for important reasoning capabilities, especially intention progression in Belief-Desire-Intention (BDI) agents. Research on GPTs has a long history but suffers from fragmentation and lack of common terminology, data formats, and enabling tools. One way to address this fragmentation is through a competition. Competitions are increasingly being used as a means to foster research and challenge the state of the art. For example, the AAMAS conference has a number of associated competitions, such as the Trading Agent Competition, while agent research is showcased at competitions such as RoboCup. We therefore issue a call for a Goal-Plan Tree Contest, with the ambition of drawing together a community and incentivizing research in intention progression

Robust Losses for Decision-Focused Learning

Optimization models used to make discrete decisions often contain uncertain parameters that are context-dependent and are estimated through prediction. To account for the quality of the decision made based on the prediction, decision-focused learning (end-to-end predict-then-optimize) aims at training the predictive model to minimize regret, i.e., the loss incurred by making a suboptimal decision. Despite the challenge of this loss function being possibly non-convex and in general non-differentiable, effective gradient-based learning approaches have been proposed to minimize the expected loss, using the empirical loss as a surrogate. However, empirical regret can be an ineffective surrogate because the uncertainty in the optimization model makes the empirical regret unequal to the expected regret in expectation. To illustrate the impact of this inequality, we evaluate the effect of aleatoric and epistemic uncertainty on the accuracy of empirical regret as a surrogate. Next, we propose three robust loss functions that more closely approximate expected regret. Experimental results show that training two state-of-the-art decision-focused learning approaches using robust regret losses improves test-sample empirical regret in general while keeping computational time equivalent relative to the number of training epochs.Comment: 13 pages, 3 figure

Towards automatic robust planning for the discrete commanding of aerospace equipment

Abstract-An increasing requirement for satellites, space probes and (unmanned) aircraft is that they exhibit robust behaviour without direct human intervention. Autonomous operation is required in spite of incomplete knowledge of an uncertain environment. In particular, embedded equipment that processes sensing data must consider uncertain input parameters while managing its own activities. We show how uncertainty may be addressed in constraint-based planning and scheduling functions for aerospace equipment, contrasting with some current practice in Integrated Modular Avionic (IMA) design. We produce a conditional plan that takes account of foreseeable contingencies, so guaranteeing system behaviour in the worst case. Executing a branch of the plan corresponds to synthesising a deterministic finite state automaton capable of discrete event commanding of an avionic sub-system. Experimental results show the feasibility of the approach for realistic aerospace equipment

Comparative analysis of knowledge representation and reasoning requirements across a range of life sciences textbooks.

BackgroundUsing knowledge representation for biomedical projects is now commonplace. In previous work, we represented the knowledge found in a college-level biology textbook in a fashion useful for answering questions. We showed that embedding the knowledge representation and question-answering abilities in an electronic textbook helped to engage student interest and improve learning. A natural question that arises from this success, and this paper's primary focus, is whether a similar approach is applicable across a range of life science textbooks. To answer that question, we considered four different textbooks, ranging from a below-introductory college biology text to an advanced, graduate-level neuroscience textbook. For these textbooks, we investigated the following questions: (1) To what extent is knowledge shared between the different textbooks? (2) To what extent can the same upper ontology be used to represent the knowledge found in different textbooks? (3) To what extent can the questions of interest for a range of textbooks be answered by using the same reasoning mechanisms?ResultsOur existing modeling and reasoning methods apply especially well both to a textbook that is comparable in level to the text studied in our previous work (i.e., an introductory-level text) and to a textbook at a lower level, suggesting potential for a high degree of portability. Even for the overlapping knowledge found across the textbooks, the level of detail covered in each textbook was different, which requires that the representations must be customized for each textbook. We also found that for advanced textbooks, representing models and scientific reasoning processes was particularly important.ConclusionsWith some additional work, our representation methodology would be applicable to a range of textbooks. The requirements for knowledge representation are common across textbooks, suggesting that a shared semantic infrastructure for the life sciences is feasible. Because our representation overlaps heavily with those already being used for biomedical ontologies, this work suggests a natural pathway to include such representations as part of the life sciences curriculum at different grade levels

Certainty Closure: Reliable Constraint Reasoning with Incomplete or Erroneous Data

Constraint Programming (CP) has proved an effective paradigm to model and solve difficult combinatorial satisfaction and optimisation problems from disparate domains. Many such problems arising from the commercial world are permeated by data uncertainty. Existing CP approaches that accommodate uncertainty are less suited to uncertainty arising due to incomplete and erroneous data, because they do not build reliable models and solutions guaranteed to address the user's genuine problem as she perceives it. Other fields such as reliable computation offer combinations of models and associated methods to handle these types of uncertain data, but lack an expressive framework characterising the resolution methodology independently of the model. We present a unifying framework that extends the CP formalism in both model and solutions, to tackle ill-defined combinatorial problems with incomplete or erroneous data. The certainty closure framework brings together modelling and solving methodologies from different fields into the CP paradigm to provide reliable and efficient approches for uncertain constraint problems. We demonstrate the applicability of the framework on a case study in network diagnosis. We define resolution forms that give generic templates, and their associated operational semantics, to derive practical solution methods for reliable solutions.Comment: Revised versio

Towards optimal demand-side bidding in parallel auctions for time-shiftable electrical loads

Increasing electricity production from renewableenergy sources has, by its fluctuating nature, created the need for more flexible demand side management. How to integrate flexible demand in the electricity system is an open research question. We consider the case of procuring the energy needs of a time-shiftable load through a set of simultaneous second price auctions. We derive a required condition for optimal bidding strategies. We then show the following results and bidding strategies under different market assumptions. For identical uniform auctions and multiple units of demand, we show that the global optimal strategy is to bid uniformly across all auctions. For non-identical auctions and multiple units, we provide a way to find solutions through a recursive approach and a non-linear solver. We show that our approach outperforms the literature under higher uncertainty conditions

Hedonic Seat Arrangement Problems

In this paper, we study a variant of hedonic games, called \textsc{Seat Arrangement}. The model is defined by a bijection from agents with preferences to vertices in a graph. The utility of an agent depends on the neighbors assigned in the graph. More precisely, it is the sum over all neighbors of the preferences that the agent has towards the agent assigned to the neighbor. We first consider the price of stability and fairness for different classes of preferences. In particular, we show that there is an instance such that the price of fairness ({\sf PoF}) is unbounded in general. Moreover, we show an upper bound $\tilde{d}(G)$ and an almost tight lower bound $\tilde{d}(G)-1/4$ of {\sf PoF}, where $\tilde{d}(G)$ is the average degree of an input graph. Then we investigate the computational complexity of problems to find certain ``good'' seat arrangements, say \textsc{Maximum Welfare Arrangement}, \textsc{Maximin Utility Arrangement}, \textsc{Stable Arrangement}, and \textsc{Envy-free Arrangement}. We give dichotomies of computational complexity of four \textsc{Seat Arrangement} problems from the perspective of the maximum order of connected components in an input graph. For the parameterized complexity, \textsc{Maximum Welfare Arrangement} can be solved in time $n^{O(\gamma)}$, while it cannot be solved in time $f(\gamma)^{o(\gamma)}$ under ETH, where $\gamma$ is the vertex cover number of an input graph. Moreover, we show that \textsc{Maximin Utility Arrangement} and \textsc{Envy-free Arrangement} are weakly NP-hard even on graphs of bounded vertex cover number. Finally, we prove that determining whether a stable arrangement can be obtained from a given arrangement by $k$ swaps is W[1]-hard when parameterized by $k+\gamma$, whereas it can be solved in time $n^{O(k)}$